Methods, devices, and systems for headlight illumination for semi-autonomous vehicles

ABSTRACT

Disclosed herein are methods, systems, and devices for providing optimized simultaneous illumination for human vision and machine based navigation vision on a semi-autonomous vehicle. In one embodiment, a system includes a first control output configured to provide first illumination control information including first active cycle times for a first illumination source. The first illumination source is configured to provide a first frequency band of illumination for machine vision navigation of the semi-autonomous vehicle. The system further includes a second control output configured to provide second illumination control information including second active cycle times for a second illumination source. The second illumination source is configured to provide a second frequency band of illumination for a human driver of the semi-autonomous vehicle. The system further includes a first monitor input configured to receive ambient illumination information from a camera system.

PRIORITY CLAIM

This application is a continuation of U.S. Application No. 17/556,138entitled “METHODS, DEVICES, AND SYSTEMS FOR HEADLIGHT ILLUMINATION FORSEMI-AUTONOMOUS VEHICLES” (Attorney Docket No. 165/13 UTIL), which wasfiled on Dec. 20, 2021, which is a continuation of International PatentApplication No. PCT/US2020/039364 entitled “METHODS, DEVICES, ANDSYSTEMS FOR HEADLIGHT ILLUMINATION FOR SEMI-AUTONOMOUS VEHICLES”(Attorney Docket No. 165/13 PCT), filed on Jun. 24, 2020, which claimsthe benefit of U.S. Provisional Pat. Application No. 62/865,546 entitled“METHODS, DEVICES, AND SYSTEMS FOR HEADLIGHT ILLUMINATION FORSEMI-AUTONOMOUS VEHICLES” (Attorney Docket No. 165/13 PROV), which wasfiled on Jun. 24, 2019, the entire contents of all are incorporatedherein by reference.

TECHNICAL FIELD

The present invention relates generally to semi-autonomous vehicles, andmore specifically, to illumination for machine vision and human visionfor semi-autonomous vehicles.

BACKGROUND

Complex camera systems are currently being deployed on semi-autonomousvehicles to provide vehicle navigation and obstacle avoidance supportfor drivers of the vehicles. The associated cameras with these systemsmust provide quality resolution with adequate frame rates to bepractical. Light sources (e.g. headlights) can allow the cameras tooperate at optimum parameters (e.g. shutter speed/exposure, depth offield, contrast and dynamic range) and provide illumination for driverswhile visibility is clear. However, weather can have a detrimentaleffect on these systems. For example rain, sleet, snow, and fog canblock the ability of these cameras to detect obstacles or the weathermay appear as one or more obstacles. Infrared (IR) light can beeffective in providing the correct illumination for the cameras todetect obstacles in weather and allow the cameras to operate at optimumparameters while detecting obstacles. However, the IR lighting does notprovide adequate lighting for the driver of the semi-autonomous vehicle.Adding additional broad-spectrum lighting for the driver in the samelocation as IR lighting renders the effectiveness of the IR lightinguseless. Also, approaching vehicle headlights and/or street lights mayrender IR lighting useless as well.

Accordingly, a need exists for devices, systems and methods forproviding optimized simultaneous illumination for human vision andmachine based navigation vision on semi-autonomous vehicles.

SUMMARY

This Summary is provided to introduce a selection of concepts in asimplified form that are further described below in the DetailedDescription. This Summary is not intended to identify key features oressential features of the claimed subject matter, nor is it intended tobe used to limit the scope of the claimed subject matter.

Disclosed herein are methods, systems, and devices for solving theproblem of providing optimized simultaneous illumination for humanvision and machine based navigation vision on a semi-autonomous vehicle.In one embodiment, a system includes a first control output configuredto provide first illumination control information including first activecycle times for a first illumination source. The first illuminationsource is configured to provide a first frequency band of illuminationfor machine vision navigation of the semi-autonomous vehicle. The systemfurther includes a second control output configured to provide secondillumination control information including second active cycle times fora second illumination source. The second illumination source isconfigured to provide a second frequency band of illumination for ahuman driver of the semi-autonomous vehicle. The system further includesa first monitor input configured to receive ambient illuminationinformation from a camera system. The ambient illumination informationis based on non-illumination cycle times. The non-illumination cycletimes, the first active cycles times, and the second active cycle timesare mutually exclusive. The ambient illumination information indicates aprobability of effectiveness of the first illumination source.

In some embodiments, the first illumination source and the secondillumination source may be housed within a standardized vehicleheadlight assembly. The camera system may also be housed within thestandardized vehicle headlight assembly. The first illumination sourcemay be further configured to provide a first lumen level between 3500lumens and 7500 lumens. The second illumination source may be furtherconfigured to provide a second lumen level between 3500 lumens and 7500lumens.

In some embodiments, the system may further include a first computingdevice. The first computing device may be configured to provide thefirst control output, the second control output, and the first monitorinput. The first computing device may also be housed within thestandardized vehicle headlight assembly. The first computing device maybe configured to receive synchronization information from a vehiclemanagement system. The non-illumination cycle times, the first activecycles times, and the second active cycle times may be based on thesynchronization information.

In some embodiments, the first frequency band of illumination and thesecond frequency band of illumination may be mutually exclusive. Incertain embodiments, the first frequency band of illumination may becentered in a range between 820 nanometers and 880 nanometers and thesecond frequency band of illumination may be centered in a range between380 nanometers and 740 nanometers.

In some embodiments, the first illumination control output may befurther configured to provide first intensity level information for thefirst source and the second control output may be further configured toprovide second intensity level information for the second illuminationsource. The first control output may be a first controlled currentsource and the second control output may be a second controlled currentsource. In other embodiments, the first control output may be a firstcontrolled voltage source and the second control output may be a secondcontrolled voltage source. The first illumination source may include afirst light-emitting-diode (LED) array and the second illuminationsource may include a second LED array.

In some embodiments, the system may also include a first camera controloutput configured to provide first camera timing information to thecamera system. The first camera timing information may include a firstcamera frame rate and a first camera per frame exposure time. The firstcamera frame rate may be approximately 24 frames per second (fps), 30fps, 60 fps, or 120 fps. The first camera per frame exposure time may bebetween 100 microseconds and 300 microseconds. In other embodiments, thefirst camera per frame exposure time may be less than 100 microseconds.In still other embodiments, the first camera per frame exposure time maybe greater than 300 microseconds. The first illumination source may befurther configured to provide the first frequency band of illuminationduring active exposure intervals of the camera system and the secondillumination source may be further configured to provide the secondfrequency band of illumination during non-active exposure intervals ofthe camera system. The first control output may be further configured tovary the first frequency band of illumination. The second control outputis further configured to vary the second frequency band of illumination.

In some embodiments, the system may also include a camera interfaceconfigured to receive a plurality of images from the camera system andthe system may be further configured to do derive the ambientillumination information from the plurality of images.

BRIEF DESCRIPTION OF THE DRAWINGS

The foregoing summary, as well as the following detailed description ofpreferred embodiments, is better understood when read in conjunctionwith the appended drawings. For the purposes of illustration, there isshown in the drawings exemplary embodiments; however, the presentlydisclosed invention is not limited to the specific methods andinstrumentalities disclosed. In the drawings:

FIG. 1 depicts a block diagram illustrating an illumination system forproviding optimized simultaneous illumination for human vision andmachine based navigation vision on a semi-autonomous vehicle inaccordance with embodiments of the present disclosure.

FIG. 2 depicts a timing diagram illustrating control signals providedfor a camera system and two illumination sources in accordance withembodiments of the present disclosure.

FIG. 3 depicts another block diagram illustrating another system forproviding optimized simultaneous illumination for human vision andmachine based navigation vision on a semi-autonomous vehicle inaccordance with embodiments of the present disclosure.

FIG. 4 depicts a block diagram illustrating a control device suitablefor the system of FIG. 1 and/or the system of FIG. 2 in accordance withembodiments of the present disclosure.

DETAILED DESCRIPTION

The presently disclosed subject matter is described with specificity tomeet statutory requirements. However, the description itself is notintended to limit the scope of this patent. Rather, the inventors havecontemplated that the claimed invention might also be embodied in otherways, to include different steps or elements similar to the onesdescribed in this document, in conjunction with other present or futuretechnologies. Moreover, although the term “step” may be used herein toconnote different aspects of methods employed, the term should not beinterpreted as implying any particular order among or between varioussteps herein disclosed unless and except when the order of individualsteps is explicitly described.

In general, this disclosure relates to semi-autonomous vehicles andimproved methods for simultaneous illumination supporting machine vision(e.g. autonomous navigation) and human vision (e.g. vehicle driver)within the semi-autonomous vehicles.

FIG. 1 depicts a block diagram 100 illustrating an illumination system102 configured for providing optimized simultaneous illumination forhuman vision and machine based navigation vision on a semi-autonomousvehicle in accordance with embodiments of the present disclosure. Theillumination system 102 may be incorporated with any type ofsemi-autonomous vehicle such as an automobile, truck, motorcycle, van,boat, plane, helicopter, or the like. The illumination system 102includes a control device 104, a camera system 106, a navigationillumination source 108, and a driver illumination source 110. Theillumination system 102 is configured to provide illumination such thatthe camera system 108 can detect obstacles such as a pedestrian 112through weather 114. The weather 114 may be rain, fog, sleet, snow,hail, smoke, smog, and/or some combination thereof. Other obstacles mayinclude other vehicles, road signs, animals, or the like.

The navigation illumination source 108 may include an infrared (IR)light emitting diode (LED) array. The navigation illumination source 108may have a maximum intensity level between 3500 lumens and 7500 lumens.Additionally, the navigation illumination source 108 may have afrequency band of illumination centered in a range between 820nanometers and 880 nanometers and optimized to penetrate weather todetect obstacles. The driver illumination source 108 may include a whiteLED array. The driver illumination source 110 may also have a maximumintensity level between 3500 lumens and 7500 lumens. A frequency band ofillumination for the driver illumination source 110 may be centered in arange between 380 nanometers and 740 nanometers and optimized for humanvision.

The control device 104 is coupled with the navigation illuminationsource 108 via a status/control interface 116. The control device 104provides navigation illumination control information via thestatus/control interface 116 to the navigation illumination source 108for machine vision for the camera system 106. The navigationillumination control information includes active cycle times for whenthe navigation illumination source 108 is on. The navigationillumination control information may also include intensity levelinformation and/or frequency band information. The control device 104may also receive status information including failures from thenavigation illumination source 108 via the status/control interface 116.

The control device 104 is coupled with the driver illumination source110 via another status/control interface 118. The control device 104provides driver illumination control information via the status/controlinterface 118 to the driver illumination source 110 for driver vision.The driver illumination control information includes active cycle timesfor when the driver illumination source 110 is on. The driverillumination control information may also include intensity levelinformation and/or frequency band information. The control device 104may also receive status information including failures from the driverillumination source 110 via the status/control interface 118.

The control device 104 is coupled with the camera system via anotherstatus/control interface 120. The control device 104 provides cameratiming information via the status/control interface 118 to the camerasystem 106. One or more cameras of the camera system 106 may eachinclude a charge-coupled device (CCD) that acts as an image sensor forcapturing high resolution images for the navigation function. In otherembodiments, each camera of the camera system 106 may include acomplementary metal-oxide-semiconductor (CMOS) sensor or an N-typemetal-oxide-semiconductor (NMOS) sensor for capturing the highresolution images. The camera timing information may include cameraframe rate and a camera per frame exposure time for one or more camerasof the camera system 106. The camera frame rate may be approximately 24frames per second (fps), 30 fps, 60 fps, 120 fps, or the like. Thecamera per frame exposure time may be between 100 microseconds and 300microseconds. In other embodiments, the camera per frame exposure timemay be less than 100 microseconds. In still other embodiments, thecamera per frame exposure time may be greater than 300 microseconds. Thestatus/control interface 120 also provides ambient illuminationinformation from the camera system 106 to the control device 104. Theambient illumination information is based on camera images capturedduring non-illumination cycle times. The camera images may each have apixel resolution of 1280×720, 1920×1080, 3840×2160, 7680×4320, or thelike.

The control device 104 also provides a status/control interface 122 forcoupling to a vehicle navigation system. The camera system 106 mayprovide the ambient illumination information and/or the camera imagesfrom the camera system 106 to the vehicle navigation system via thestatus/control interface 122. In some embodiments, the status/controlinterface 122 may be an Ethernet interface. For example, the Ethernetinterface may be GigE, Dual GigE, 5GigE, 10 GigE, or the like. In otherembodiments, the status/control interface 122 may be a Camera Link HSinterface, a CoaXPress® interface, a Universal Serial Bus (USB) 3.0interface, or the like.

The control device 104 may be any computing device (e.g.microcontroller) suitable for the semi-autonomous vehicle. The controldevice 102 may also be an application-specific integrated circuit(ASIC), a field-programmable gate array (FPGA), discrete logic/circuity,or a combination thereof. In some embodiments, the control device 104may be implemented within the vehicle navigation system. In otherembodiments, the control device 104 may be implemented within the camerasystem 106, the navigation illumination source 108, and/or the driverillumination source 110.

FIG. 2 depicts a timing diagram 200 illustrating control signals 202,204, and 206 provided within the illumination system 102 of FIG. 1 inaccordance with embodiments of the present disclosure. The controlsignal 202 is provided from the control device 104 to the navigationillumination source 108 via the status/control interface 116. Thecontrol signal 202 may be provided by a controlled current source or acontrolled voltage source of the control device 104. The control signal204 is provided from the control device 104 to the driver illuminationsource 110 via the status/control interface 118. The control signal 204may be provided by a controlled current source or a controlled voltagesource of the control device 104. The control signal 206 is providedfrom the control device 104 to the camera system 106 via thestatus/control interface 120. The control signal 206 may be provided bya controlled current source or a controlled voltage source of thecontrol device 104. In other embodiments the control signal 206 may beprovided from the camera system 106 to the control device 104 via thestatus/control interface 120.

The control signal 202 provides both active cycle (i.e. the time betweena rising edge and a falling edge of the waveform) for when thenavigation illumination source 108 is on and a navigation luminanceintensity level based on a voltage value or a current value. The controlsignal 204 provides both active cycle for when the driver illuminationsource 110 is on and a driver luminance intensity level based on avoltage value or a current value. The control signal 208 provides anindication for a camera exposure time and camera frame rate. Thenavigation illumination source 108 is cycled on and off, and the driverillumination source is cycled on and off such that one or more camerasof the camera system 106 can capture an image with no illumination andcapture an image with just the navigation illumination source 108 on.The image with no illumination is used to determine ambient illuminationinformation and the image with the navigation illumination source 108 onis used for obstacle detection. If the ambient illumination informationindicates high ambient illumination levels at or near a possibleobstacle a vehicle navigation system may determine the obstacledetection may be inaccurate. Or, if the ambient illumination informationindicates overall illumination levels too high (e.g. approachingheadlights) the vehicle navigation system may suspend obstacle detectionuntil the overall illumination levels drop to an acceptable level toallow the navigation illumination source 108 to be effective. In someembodiments, one or more of the control signals 202, 204, and 206 may bederived from synchronization information received from the vehiclenavigation system.

FIG. 3 depicts another block diagram 300 illustrating another system forproviding optimized simultaneous illumination for human vision andmachine based navigation vision on a semi-autonomous vehicle inaccordance with embodiments of the present disclosure. A vehiclenavigation system 302 is coupled with a left headlight assembly 304A viaa status/control interface 122A. The vehicle navigation system 302 isalso coupled with a right headlight assembly 304B via a status/controlinterface 122B. The left headlight assembly 304A and the right headlightassembly 304B may each be dimensioned as a standardized headlightassembly for a given vehicle. The illumination system 102 of FIG. 1 isincorporated into the left head light assembly 304A and includes acontrol device 104A, a camera system 106A, a navigation illuminationsource 108A, a driver illumination source 110A, and status/controlinterfaces 116A, 118A, and 120A. The illumination system 102 of FIG. 1is also incorporated into the right head light assembly 304B andincludes a control device 104B, a camera system 106B, a navigationillumination source 108B, a driver illumination source 110B, andstatus/control interfaces 116B, 118B, and 120B. Synchronizationinterfaces 306A and 306B may also be used to provide one or more precisetiming control signals between control devices 104A and 104B.

FIG. 4 depicts a block diagram 400 illustrating one embodiment of thecontrol device 102 of FIG. 1 and FIG. 3 for implementing the previouslydisclosed methods of simultaneous illumination supporting machine vision(e.g. autonomous navigation) and human vision (e.g. vehicle driver)within semi-autonomous vehicles. The control device 102 includes aprocessor 402 and a memory 404. In certain embodiments themicroprocessor may be an 8-bit, a 16-bit, a 32-bit, or a 64-bitarchitecture processor. The memory 404 may include a combination ofvolatile memory (e.g. random access memory) and non-volatile memory(e.g. flash memory). In certain embodiments, the memory 404 may beintegrated with the processor 402. In some embodiments the controldevice 102 may be integrated at least partially within amicrocontroller. The microcontroller may include timers, signalgenerators, parallel input/output (I/O), and serial I/O. Additionallythe microcontroller may include analog-to-digital converters anddigital-to-analog converters.

The control device 102 includes status indicators 406. The statusindicators may include one or more light emitting diodes (LEDs) thatindicate power, correct operation, and/or a failed state. The controldevice 102 also included the navigation illumination status/controlinterface 116, the driver illumination status/control interface 118, thecamera system status/control interface 120; the vehicle navigationsystem status/control interface 122, and the synchronization interface306 previously disclosed in the systems of FIG. 1 and FIG. 3 . Incertain embodiments, one or more of these interfaces may be configure toconnect with a controller area network (CAN).

The various techniques described herein may be implemented with hardwareor software or, where appropriate, with a combination of both. Thus, themethods and apparatus of the disclosed embodiments, or certain aspectsor portions thereof, may take the form of program code (i.e.,instructions) embodied in tangible media, such as floppy diskettes,CD-ROMs, hard drives, or any other machine-readable storage medium,wherein, when the program code is loaded into and executed by a machine,such as a computer, the machine becomes an apparatus for practicing thepresently disclosed invention. In the case of program code execution onprogrammable computers, the computer will generally include a processor,a storage medium readable by the processor (including volatile andnon-volatile memory and/or storage elements), at least one input deviceand at least one output device. One or more programs are preferablyimplemented in a high level procedural or object oriented programminglanguage to communicate with a computer system. However, the program(s)can be implemented in assembly or machine language, if desired. In anycase, the language may be a compiled or interpreted language, andcombined with hardware implementations.

The described methods and apparatus may also be embodied in the form ofprogram code that is transmitted over some transmission medium, such asover electrical wiring or cabling, through fiber optics, or via anyother form of transmission, wherein, when the program code is receivedand loaded into and executed by a machine, such as an EPROM, a gatearray, a programmable logic device (PLD), a client computer, a videorecorder or the like, the machine becomes an apparatus for practicingthe presently disclosed invention. When implemented on a general-purposeprocessor, the program code combines with the processor to provide aunique apparatus that operates to perform the processing of thepresently disclosed invention.

While the embodiments have been described in connection with thepreferred embodiments of the various figures, it is to be understoodthat other similar embodiments may be used or modifications andadditions may be made to the described embodiment for performing thesame function without deviating therefrom. Therefore, the disclosedembodiments should not be limited to any single embodiment, but rathershould be construed in breadth and scope in accordance with the appendedclaims.

What is claimed:
 1. A system for providing simultaneous illumination formachine vision and human vision, the system comprising: a first controloutput configured to provide first illumination control informationincluding first active cycle times for a first illumination source,wherein the first illumination source is configured to provide a firstfrequency band of illumination for a camera system; a second controloutput configured to provide second illumination control informationincluding second active cycle times for a second illumination source,wherein the second illumination source is configured to provide a secondfrequency band of illumination for a human; and a first monitor inputconfigured to receive ambient illumination information from the camerasystem, wherein: the ambient illumination information is based onnon-illumination cycle times; the non-illumination cycle times, thefirst active cycles times, and the second active cycle times aremutually exclusive; and the ambient illumination information indicates aprobability of effectiveness of the first illumination source.
 2. Thesystem of claim 1, wherein the first illumination source and the secondillumination source are housed within a standardized illuminationassembly.
 3. The system of claim 2, wherein the camera system is housedwithin the standardized illumination assembly.
 4. The system of claim 3further comprising a first computing device, wherein: the firstcomputing device is configured to provide the first control output, thesecond control output, and the first monitor input; and the firstcomputing device is housed within the standardized illuminationassembly.
 5. The system of claim 4, wherein the first computing deviceis configured to: receive synchronization information from a managementsystem; and provide the non-illumination cycle times, the first activecycles times, and the second active cycle times based on thesynchronization information.
 6. The system of claim 1, wherein the firstfrequency band of illumination and the second frequency band ofillumination are mutually exclusive.
 7. The system of claim 1, whereinthe first control output is further configured to provide firstintensity level information for the first illumination source.
 8. Thesystem of claim 7, wherein, the second control output is furtherconfigured to provide second intensity level information for the secondillumination source.
 9. The system of claim 1, wherein the firstillumination source includes a first light-emitting-diode (LED) arrayand the second illumination source includes a second LED array.
 10. Thesystem of claim 1 further comprising a first camera control outputconfigured to provide first camera timing information to the camerasystem.
 11. The system of claim 10, wherein the first camera timinginformation comprises a first camera frame rate and a first camera perframe exposure time.
 12. The system of claim 11, wherein the firstcamera frame rate is approximately 24 frames per second (fps), 30 fps,60 fps, or 120 fps.
 13. The system of claim 11, wherein the first cameraper frame exposure time is between 100 microseconds and 300microseconds.
 14. The system of claim 11, wherein the first camera perframe exposure time is less than 100 microseconds.
 15. The system ofclaim 11, wherein the first camera per frame exposure time is greaterthan 300 microseconds.
 16. The system of claim 1 wherein the firstillumination source is further configured to provide the first frequencyband of illumination during active exposure intervals of the camerasystem.
 17. The system of claim 16, wherein the second illuminationsource is further configured to provide the second frequency band ofillumination during non-active exposure intervals of the camera system.18. The system of claim 1, wherein the first frequency band ofillumination is centered in a range between 820 nanometers and 880nanometers and the second frequency band of illumination is centered ina range between 380 nanometers and 740 nanometers.
 19. The system ofclaim 1, wherein the first control output is further configured to varythe first frequency band of illumination and the second control outputis further configured to vary the second frequency band of illumination.20. The system of claim 1, wherein the first illumination source isfurther configured to provide a first lumen level between 3500 lumensand 7500 lumens and the second illumination source is further configuredto provide a second lumen level between 3500 lumens and 7500 lumens.